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Abstract 

An elementary proof of the attainability of random coding exponent 
with linear codes for additive channels is presented. The result and proof 
are from Hamada (Proc. ITW, Chendu, China, 2006), and the present 
material explains the proof in detail for those unfamiliar with elementary 
calculations on probabilities related to linear codes. 



1 Introduction 

In this material, the details of the proof of a result in yj , an article prepared for 
an invited talk, are presented without assuming any prerequisite knowledge. In 
fact, when the author prepared the manuscript ^5], which includes one illustra- 
tive application of the method of concatenating 'conjugate code pairs' devised in 
[TJ |3] , the author thought some (or most) proofs are elementary and straightfor- 
ward, so that they are not needed for those working in our society of information 
theory. However, in this article, still more details will be presented to increase 
the accessibility. 

We remark the result and its detailed proof are written so that they can be 
read without referring to [T| . Specifically, in this material, an elementary proof 
of the attainability of random coding exponent with linear codes for additive 
channels is presented. (Of course, many proofs for the attainability of random 
coding exponent had existed, but the incentive for developing this approach 
was to design quantum error-correcting codes and codes that can be used in 
cryptographic protocols. For these purposes, we needed to design codes and 
decoders under constraints arising from quantum mechanics.) 

Thus, this material is supplementary to [1] for those unfamiliar with the 
elementary approach adopted in [1], but the result treated in this material is 
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compact, classical, and comprehensible without understanding the main issues 
treated in [1] . This approach is nothing special, but it may be said to be that of 
the method of types [HIS], which requires no prerequisite knowledge, with the 
very basics of linear codes incorporated. 

The aforementioned illustrative application of the method for concatenation 
is construction of pairs of linear codes {L\,L2) with C Li ('conjugate code 
pairs') that achieve a high information rate on the Shannon theoretic criterion. 
Such a code pair can be viewed as a succinct representation of the corresponding 
quantum error-correcting code (QECC). The code construction is exphcit in the 
standard sense that the codes are constructible with polynomial complexity. 
Another (cryptographic) application, which reflects the original motivation of 
[H [2] has been presented in [6] . 

2 Corrections and Remark to 

2.1 Corrections to jT]; Some Apply Also to 

1. p. 149, right column, line 14, 'ensemble' should be followed by '(multiset)' 

2. p. 150, left column, line —1, 

should read 

3. p. 150, right column, line —9, 'parameter /c' should read 'the number k/n' 

4. p. 151, left column, line -8, '{yf ■ ■ ■ j/j^V should read '(y^^) • • • y(^))' 

5. p. 152, left column, line 1, '(0^1 ^ 0i=i ^2'^)' should read 

6. p. 152, left column, Eq. (6), 

MQicf \ {04) < (|K(F,)| - l)g-"(i-'-^)A 

should read 

MQicf \ {0„}) < i\Vni¥,)\ - l)q--^'~^^^\T^\A 

Essentially the same errors as in 1, 2 and 6 exist in Section 4 of [2] (ver. 2), 
but the contents of Section 4 of [2] are presented below in the corrected form. 

2.2 Remark to [U [2] 

Note that, in |1] [2], an ensemble has been represented as a multiset, which is 
similar to a usual set but permits duplicated entries. 

Now the author thinks representing an ensemble as an ordered set is more 
natural, as will be done in the present article. 
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3 Preliminaries 



In this section, we fix our notation, and recall some notions to be used. As 
usual, [a\ denotes the largest integer a' with a' < a, and \a\ = — [—a\ . An 
[n, k] linear (error-correcting) code over a finite field Fg, the finite field of q 
elements, is a fc-dimensional subspace of . The dual of a linear code C C F^ 
is {y € F^ I Vcc G C, X ■ y = 0} and denoted by C^, where x ■ y = xy^ with 
being the transpose of y. The zero vector in F^ is denoted by 0„. The n x n 
identity (resp. zero) matrix is denoted by /„ (resp. 0„). For integers i < j, we 
often use the set fl Z = {i,i + 1, . . . , j}, which consists of integers lying in 
the interval [a, &] = {z G M | a < z < 6}. 

We denote the type of a; G F^ by P^, [U [5] . This means that the number of 
appearances of u G F, in x G is nPx{u). The set of all types of sequences 
in F^ is denoted by 7'„(Fg). Given a set C C F^, we put Mq{C) = \{y G 
C I Pj, = Q}\ for types Q G Vn{¥q). The list of numbers {Mq{C))q^-p„{¥,) 
may be called the P-spectrum (or simply, spectrum) of C. For a type Q, we 
put 75 = {y G F^ I P^, = Q}. We denote by V{y) the set of all probability 
distributions on a set y. The entropy of a probability distribution P on y is 
denoted by H{P), viz., H{P) = '}2,y^y —P{y) \ogP{y). Throughout, logarithms 
are to base q. 

We follow the convention to denote by Px the probability distribution of a 
random variable X. 

4 Good Codes in a Balanced Ensemble 
4.1 Balanced Ensemble 

We can find good codes in an ensemble if the ensemble is 'balanced' in the 
following sense. Suppose S = {C(*)}^i is an ensemble (ordered set) of subsets 
of F^. If there exists a constant V such that |{z G [1, A/'] n Z | x G C^^}\ = V 
for any word a: G F^ \ {0„}, the ensemble S is said to be balanced. (We remark 
that the 'balancedness' is defined in a different manner in [7] for ensembles of 
encoders, not codes.) 

The first task in [T] was to construct a relatively small balanced ensemble. 
This result can be found in [1] [5] , but it is included in Appendix IA.21 With 
the method of types, we will show that a large portion of a balanced ensemble 
consists of good codes. While the goodness of codes should be evaluated by the 
decoding error probability, it is also desirable to quantify the goodness in such a 
way that the goodness does not depend on characteristics of channels. In view 
of this, the following proposition is useful. 

The next proposition relates the spectrum of a code with its decoding error 
probability when it is used on an additive memoryless channel. 

Proposition 1 [8j Theorem 4] . Suppose we have an [n, k\ linear code C over 
Vq such that 

Mq{C) < anq''-^\T^l Q G 7'„(Fg) \ {Po„} 
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for some a„ > 1. Then, its decoding error probability with the minimum entropy 
syndrome decoding is upper-bounded by 

for any additive channel W of input-output alphabet ¥q, where r — n/n and 
£'r(VF, r) is the random coding exponent of W defined by 

E,{W,r)= min [D{Q\\W) ^ \l - r - H{Q)\+]. 

Here, D and H denote the relative entropy and entropy, respectively, and = 
max{0, x}. 



For a poof, see Section 14731 In the simplest case where g = 2, the premise of 
the above proposition reads 'the spectrum of C is approximated by the binomial 
coefficients \Tq\ up to normalization.' 

The following lemma shows a large portion of a balanced ensemble {C^*-'}fl]^ 
is made of good codes (we have applied this fact to ensembles written as 
{Cf }fJi in [1, 2 ). 

Lemma 1 IH p. 152, left column]. Assume we have a balanced ensemble 
1^(4) jjv^^ Lef us say an [n, k] code C'*-* is A-good if 

Mq{C^^) < A{\Vn{¥,)\ - l)<z-"(i-'')|75| (1) 

for all Q G 7^n (Fg) \ {Po„}j where p — Hi/n. Then, the number of codes that are 
not q^^-good in {C^'-*}^]^ is at most 

z=[Ar*g-^"J. (2) 

This lemma will be proved in Section Note, owing to Proposition [1] for 
the g'^"-good codes C*^'^ in the above lemma, the decoding error probability is 
upper-bounded by 

a'^q-r^[E.{W,p)^e]^ (3) 

where I'PnllFg)!^ is at most polynomial in n. 
4.2 Proof of Lemma [T] 

A proof of Lemma [T] will be given, though it may be a routine in information 
theory. We have a lemma. 

Lemma 2 Assume S and W are finite sets, and non-negative numbers fw{x) 
are associate with each pair {x,w) G S x W. Denote by f^ the average of fw{x) 
over S; ^ 

Then, for any a > 0, the number of members in S that fail to satisfy the condition 

Vu;gW, f^{x)<JjW\a 
is upper-bounded by a^^|S|. 
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Proof. Let X be a random variable uniformly distributed over S. Then, the 
probability that X fails to satisfy Ww g W, fw{X) < /^|W|a' is upper-bounded 
as follows: 

Pr{3weW, UiX)>7jW\a} 

W 

- E Pr{^(^) > |W|7^,a} 
(") ^ 

< E (|W|«)"'<«"'- (4) 

w:J^>0 

where the equality (i) and inequality (ii) follow from the fact that = 
implies fw{x) = /^,|W|a = for all x S S, and Markov's inequality, respectively. 
Markov's inequality is included at the end of this subsection with a proof. The 
lemma immediately follows from (j4|). □ 

Proof of LemmaUl From the fact that {C^^^jflj^ is balanced, it follows 

N* 

-^E^Qi^''') = ^I'^qI < ^ITq-I (5) 

for any Q G 7'„(Fg), Q ^ Po„. To see this, let V be the number of appearances 
of any fixed nonzero word in enumerating codewords in C^*\ i G [1, A^*] n Z. 
Then, we have trivial equalities V{q" — 1) = N*{q'^ — 1) and 

N' 

1=1 

for any Q G T'„(Fg), Q ^ Po„0 From these, we readily obtain the equality 
and hence the inequality in ([5])- Now Lemma [T] follows upon applying Lemma [2] 
to S = {(CW,i) I i G [l,iV*] n Z}, where fn,{{C,i)) = Mq{C), w = Q and 
>V-7'„(F,)\{Po„}. □ 



Lemma 3 (Markov's Inequality) For a positive constant A, and a random 
variable Y that takes non-negative values and has a positive mean fi, we have 

Pr{Y > An] < l/A. 

Proof We have fi = E^Py{y)y > Ey:y>,A My)y > E,:,>M^Y(y)M 

= IJ-A'^2, y j^PyIh) = ^APr{Y > Afi}, which implies the lemma. □ 



^The relation V(q" — 1) = N*{q'^ — 1) immediately follows by counting the pairs (x,C) 
such that X g C\{On} and C is a component of {C^'^}^-^ in two ways, and the other equality 
follows similarly. 
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4.3 Proof of Proposition [T] 

We use the following basic inequality [H [5J [5] : 

J2 ^"(y) ^ q-''™^^ (6) 

for any P G P{¥q). (Recall P" denotes the product of n copies of P.) The sym- 
metric group on {1, . . . , n}, which is composed of all permutations on {1, . . . , n}, 
is denoted by iS„. We define an action of iS„ on F^' by 

tt{{xi, . . . ,Xn)) = (a;^(i),...,a;^(„)) 

for any vr G 5„ and (xi, . . . , Xn) G F^, and put 

7r(C) = {7r(a;) I a; e C}, tt G 5„, C C F^. 

The expectation operation with respect to a random variable X taking values in 
X is denoted by Ex: 

Ex/(X) = ^ Px{x)f{x) 

where / is a real- valued function on X . 

Lemma 4 Assume a linear code C C F^ satisfies 

Mq{C \ {Qn})/\T^\ < a.nq--^, Q G P„(F,) 

with some real numbers a„ > 1 and T . Let J be a set of coset representatives 
for F^/C such that each coset D G F^/C has a representative that belongs to J 
and that attains the minimum of H^P^), x G D (the resulting decoding is called 
minimum entropy decoding). Then, we have for any P„ G 'P(F^'), 

Q6P„(F,) 

where c denotes complement, — max{i, 0}, and the random variable tt is 
uniformly distributed over iS„ . 

Corollary 1 Assume for a linear code C C F^\ Mq{C\ {0„}) is bounded as in 
Lemma^ Then, with J as in the lemma, we have for any P G P{¥q), 

where 

E{P,T) - ^ mill \D{Q\\P) + |T - H(Q)|+]. 
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A proof of Lemma m is given in the next subsection. 

Proof of Corollary^ Clearly, E7rP"(7r( 7)==) = P^iJ"^). Then, inserting the 
estimate of P"{Tq) in ^ into the bound on E7rP"(7r( J)'^) in the lemma, we 
have 

Qe-P„(F,) 

and hence, the corollary. 

Putting T = 1 — n/n in this corollary, we readily obtain the proposition. 



4.4 Proof of Lemma [4] 

In the proof, Pn{^q) is abbreviated as Vn- We will show that G — E7rP„(7r( J)"^) 
is bounded above by the claimed quantity. 

Imagine we list up all words in 7r(C \ {0„}) for all tt G iS„ permitting du- 
plication. Clearly, the number of appearances of any fixed word y e F^' in the 
list only depends on its type Py ^ Vn- Namely, for any Q Vn, there exists a 
constant, say Lq, such that 

\{7reSn\yen{C\{0n})}\^LQ (7) 

for any word y with Py = Q. Then, counting the number of words of a fixed 
type Q in the list in two ways, we have \Tq\Lq — |5„|A/q(C \ {0„}). Hence, 
for any type Q £ Vn{Pq) 

Lq _ Mq(C\{0„}) ^ _„r 



by assumption. From (jT]) and ([8]), we have 



< a„g-"' (8) 



|A,(C\{04)| 



|<5n| 

for any y ^W^, where 

Ay{C \ {On}) ^{neSnlye Trie \ {On})} 

Then, we have 



< a„g-"^ (9) 



G 



Since x ^ J occurs only if there exists a word u E F^' such that H{Pu) < -ff (Pa;) 
and w — a; G tt{C \ {0„}) from the design of J specified above (minimum entropy 
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decoding), it follows 

|{^ eS,,\xi J}|/|5„| 

< |A„-.(C\{0„})|/|5„| 

Q'ePn:H(Q')<-fi"(Px) 

< E ang"^^'^'^""^ (11) 

Q'(iV„:H(Q')<H(P^) 

where we have used ([9]) for the second inequality, and another well-known in- 
equality n [3 in] 

VQ G n.(F,), \r^\ < g"^^'^) (12) 

for the last inequality. Then, using the inequalities min{at, 1} < amin{t, 1} and 
min{s + t^l} < min{s, 1} -I- min{t, 1} for a > 1, s, i > 0, we can proceed from 
PH)) as follows, which completes the proof: 

G < E^«(^)"^K E a„g"^(«')-"^, 1 } 

^GF^' Q'eV„:H{Q')<HiP^) 

< a„ E ^"(TQ)min{ E 

QeT',! Q'eV„:H{Q')<H{Q) 

< a» E ^"C^') E min{g-"[^-^(«')l, 1 } 

QeV„ Q'eV„:H{Q')<H{Q) 

< a„\Vn\ E Pr^iTS) max g-«|T-ff(Q')|+ 



a„in.i E ^^«(r^)<?-"i^-^('3)i 



+ 



5 Concluding Remarks 

In [U [3] (or [^), quantum- mechanically compatible pairs of linear codes that 
are constructible with polynomial complexity were presented. The Calderbank- 
Shor-Steane quantum codes corresponding to the constructed pairs achieve the 
so-called Shannon rate. The most novel result among these would be the method 
for concatenating compatible (conjugate) code pairs, which have been published 
in E]. 

The present material was prepared for explaining the results not included 
in [3] for those unfamiliar with the elementary combinatorial approach (the 
method of types with the very basics of linear codes incorporated) . 

This material might be included somewhere else (possibly in some other 
context). 
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A Some Other Contents of [T] 



A.l Compatible (Conjugate) Code Pairs |T] 

Consider a pair of linear codes (Ci, C2) satisfying 

Ci C Ci, (13) 

which condition is equivalent to C C2. The following question arises from 
an issue on quantum error correction: How good both Ci and C2 can be under 
the constraint ([T3|) ? This is the subject treated in [Tl[3l[2]- 

We have named a pair (Ci,C2) with (fTS]) a conjugate code pair in [1]. In 
what follows, we will use a 'compatible code pair' in place of 'conjugate code 
pair.' 

A. 2 Code Ensemble Based on Extension Field [T] 

The companion matrix of a polynomial f{x) — — fn-ix""^^ _ . . . — f-^x — /o, 
which is monic (i.e., of which the leading term has coefficient 1), over is 
defined to be 

On-l /o 

/i 

In~l : 

fn-l 



T = 



Let T be the companion matrix, or its transpose, of a monic primitive poly- 
nomial of degree n over F^. Given an n x n matrix M, let Mj™ (resp. M|m) 
denote the m x n submatrix of M that consists of the first (resp. last) m rows 



.... = {xT'l''^ I X e Fji} and C^'^ = {x{T-'Y\k^ | x G F^ j 

i = 1, 2, . . . , where M* denotes the transpose of M. Then, setting 

B = Bt = {(C«,C«)}£-\ (14) 

we have the next lemma. 

Lemma 5 [U Lemma 1] . Let T he the companion matrix of a monic primitive 
polynomial of degree n over ¥q. For integers ki , k2 with 0<n — fc2<A:i<n and 

Bt = constructed as above, any (Cj;*"*, Cj''') is a compatible 

code pair, and both {Cf^}£r^ and {C^'^jCi^ are balanced. 

Remark. It is known (and proved in a self-contained manner in [3l, Sec- 
tions VII]) that the matrix T has the following property, which are used in the 
proof of Lemma [5] below: The set {On, /„, T, . . . , T"^ is isomorphic to F^™ 
as a field. □ 

Proof of Lemma[S[^. The condition ([T^ is fulfilled since T'^T~^ — In implies 
that the Ca'^-"- is spanned by the first n — k2 rows of T*. (This is easily seen if 
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Figure 1: A basic structure of an [[n, k]] compatible code pair. 



we divide the two matrices on the left-hand side of T^T ' = /„ into submatrices 
as in Figure [1]) 

We can write C^*^ = {yT^ I 2/ £ ^q, supp y C [1, fciJnZ}, where supp (yi, . . . , y„) 

= {* I 2/i 7^ 0}- Imagine we list up all codewords in cj*'' permitting duplica- 
tion. Specifically, we list up all yT* as y and i vary over the range {y \ y G 
Fg, suppy C [1, ki] n Z} and over [1, — 1] n Z, respectively. 

With y £ \ {0} fixed, yT*, i e [1, q" - 1] nZ, are ah distinct since ^ 
implies yT* — yT^ = yT' for some / and yT' is not zero. Hence, any nonzero 
fixed word in appears exactly q'^^ — 1 times in listing yT' as above. Namely, 

the ensemble {Ci^}1_^^ is balanced. Using (T^')' in place of T\ we see the 
ensemble {C2^}1_^^ is also balanced, completing the proof. □ 

Lemmas [T] and show the existence of a compatible code pair having expo- 
nentially decreasing decoding error probabilities in B. 
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